paper unified graph structured model
Google Research's SOTA GNN 'Reasons' Interactions over Time to Boost Video Understanding
In the last few years, deep learning driven computer vision (CV) research has achieved impressive progress on classifying video clips taken from the Internet and analyzing the human actions therein. Such video-based tasks are challenging, as they require an understanding of the interactions between humans, objects and other content and context within a given scene, as well as reasoning over long temporal intervals. A successful CV model in this area needs to capture both spatial and long-range temporal interactions while also being "intelligent" enough to reason based on its observations. In the paper Unified Graph Structured Models for Video Understanding, a Google Research team proposes a message-passing graph neural network (MPNN) that can explicitly model these spatio-temporal relations, use either implicitly (with supervision) or explicitly (without supervision) captured representations of objects, and generalize previous structured models for video understanding. The Google Research paper Unified Graph Structured Models for Video Understanding focuses on spatio-temporal action recognition and video scene graph parsing, which require reasoning about interactions between actors, objects and their environment in both space and time.